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Attachment X 


Evaluation of the Pilot Project 


1. As a basis for evaluation, there are (a) the observed scores 

and performance of individuals and teams in the exercises, 

(b) the perceptions and feelings of most of the participants, 
as reported in their answers to the quest ionnaire, and (c) the 
views of the Director of Finance and seven division and staff 
chiefs, as set forth in his memorandum of 29 June 1966 to the 
Deputy Director for Support (Subject: '’Evaluation of the Office 
of Finance Pilot Project"). Unfortunately, neither quantitative 
data on production nor critical incidents could be collected to 
augment the findings. 


2* The crucial question is this: did the Grid training cause 

changes which have led to greater effectiveness and efficiency in 
the Office of Finance ? In answer to this question, the Management 
Training Faculty has reached the following conclusions: 

a. Working relationships among people . The greatest positive 
change took place in this area. There is now more team- 
work, better understanding, and a sounder basis for problem 
solving. 

b. Communication . Every team in Phase I listed this as a 
major problem in the Office of Finance. Many individuals 
and units have now increased their ability to communicate. 
This is a clear gain. 

c. Productivity . The evidence is inconclusive. The Director 
of Finance has not observed any quantitative increase. 

Four of his division chiefs feel that there has been a 
measurable improvement in the quality or quantity of 
work production; three see none. In the questionnaire, 

22 supervisors said there has been a significant increase 
in the amount of work turned out by their subordinates; 
but 40 did not see this. About half of the officers see 
a significant increase in the general effectiveness of the 
work teams they belong to. Creativity was identified as 
a serious problem during Phase I; again about half of the 
supervisors have now seen a significant increase in the 
creativity of their subordinates. Evidently the Grid did 
not help some units, but they may have seen themselves as 
fully effective before the Project, and therefore no 
change . 
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d. Goal Setting. The results of this key aspect of the 
Project (in effect, "management by objectives") are 
disappointing. Some teams were not very successful 

in identifying specific meaningful goals for themselves. 

Only 26# of the replies on the questionnaire say that 
significant progress has been made toward these goals, 
and only 15# say that their team is currently following 
a systematic plan for improvement. This 15# (9 people) 
may possibly all be members of the same unit. The 
Director of Finance mentioned an increased ability and 
desire to set team goals as a positive benefit of the 
Project, but only one out of his seven division and staff 
chiefs made any mention of this activity. 

e. Phase I. The performance of the Office in Phase I was 
outstanding in terms of effort, commitment, and accomplish- 
ment. Working with these officers was a highly gratifying 
experience . 

f. Phase II. This was not as successful as Phase I in terms 
of results and enthusiasm, partly because of the time lag 
and other errors previously noted, but primarily because 
it is a harder exercise. Phase II is not concerned with 
artificial problems and school theory, but with real life 
and the solution of actual managerial problems. Although 
some participants were enthusiastic about it and recognized 
its potential, the majority apparently found it difficult 
and somewhat unsatisfactory. A few were completely 
negative. The Director of Finance feels that the Office 
received considerably more benefit from Phase I than from 
Phase II. Of his division chiefs, two rated Phase I as 
"excellent" and five rated it as "good." Two rated 

Phase II as "excellent," four rated it as "good," and one 
as "fair." Some of their remarks indicate inconplete 
comprehension of Phase II; it should be added that the 
training officers also had less than couplets understanding 
of Phase II at that time. 


3. This being the case, and with all the other negative factors 

operating, the Management Training Faculty feels that the results 
obtained were remarkable and very much to the credit of the 
participants in this Project. It seems clear that the results are 
not conclusive, in that a number of the findings are quite 
favorable and others are unfavorable; some individuals and units 
apparently gained maximum benefit, while others gained none. 

Future Grid projects will benefit from the experience gained from 
this pioneer effort. 
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Cost . The Management Training Faculty feels that the 
Director of Finance gives a distorted picture of the cost of the 
Project by adding the salaries of the participants to the cost 
of consultants and materials. The recorded expenditures were 
$17,000, considerably less than the $25,000 originally estimated. 


5* Dilution of the Effort . One negative factor which has been 

operating to lessen the effect of the training must not be over- 
looked. Rotation of personnel has brought into the Office a 
number of individuals who have not had the Grid. As a result, 
some units are split between “Grid-speakers 11 and "non-Grid- 
speakers,* 1 a situation not conducive to mutual understanding and 
the preservation of gains made during the Project. 
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